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Abstract 

The Traveler's Dilemma game and the Minimum Effort Coordination game are two social dilemmas that have attracted 
considerable attention due to the fact that the predictions of classical game theory are at odds with the results found when 
the games are studied experimentally. IVloreover, a direct application of deterministic evolutionary game theory, as 
embodied in the replicator dynamics, to these games does not explain the observed behavior. In this work, we formulate 
natural variants of these two games as smoothed continuous-strategy games. We study the evolutionary dynamics of these 
continuous-strategy games, both analytically and through agent-based simulations, and show that the behavior predicted 
theoretically is in accord with that observed experimentally. Thus, these variants of the Traveler's Dilemma and the 
Minimum Effort Coordination games provide a simple resolution of the paradoxical behavior associated with the original 
games. 
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Introduction 

Social dilemmas embody the tension between individual self- 
interest and the common good that is inherent in many important 
situations in the real world. In a social dilemma, individually 
reasonable behavior results in a situation in which all individuals 
are less well off than they could otherwise have been [1]. Social 
dilemmas underlie many of the most fundamental and intractable 
problems in the biological and social sciences, such as the 
evolution of cooperation [2] and the efficient use of limited shared 
resources [3] . From a more formal point of view, a social dilemma 
can be modeled as a game in which there exists at least one Hicks 
inefficient Nash equilibrium. It is Hicks inefficient (i.e. socially 
inefficient) in that there is at least one other outcome in which all 
individuals would be better off, and since it is a Nash equilibrium 
there is no incentive for any individual to change their behavior 
[1]. Examples of 2-person games that are social dUemmas include: 
The Prisoner's Dilemma [2,4,5], the Snowdrift game (also known 
as the Chicken or Hawk-Dove game) [6] , and the Stag-Hunt game 
(also known as the Assurance game) [7]. Multi-person social 
dilemmas include, the Public Goods game [8,9] and the Tragedy 
of the Commons [3] . WhUe the original game theory formaliza- 
tions of these social dilemmas typically involved discrete-strategy 
games, more recently continuous-strategy versions of the Prison- 
er's Dilemma game [10], the Snowdrift game [1 1], the Tragedy of 
the Commons [12] game, and the Public Goods game [13] have 
been formulated and studied. 

One particularly fascinating class of social dilemmas are those 
for which the predictions of game theory appear to be inconsistent 



with the behavior observed when the games are played 
experimentally [14]. The Traveler's Dilemma (TD) game [15- 
17] and the Minimum Effort Coordination (MEC) game [17,18] 
are two celebrated examples of such games. 

These games challenge the notion that the rational solution to a 
game, as embodied in the concept of the Nash equilibrium, 
accurately describes the behavior of humans engaged in these 
social dilemmas. In the TD game there exists a unique, pure 
strategy, Nash equilibrium that is undesirable for all concerned. In 
contrast to the TD game, in the MEC game every pure strategy is 
a Nash equilibrium, and thus the rational solution concept lacks 
any prescriptive or predictive power. 

The TD game is conventionally introduced through a story of 
the following form. Two travelers, on their return journey from an 
exotic country, find that their luggage containing identical 
souvenirs has been lost by the airline. The officer in the claims 
department puts them in separate rooms, hands each of them a 
claims form, and tells them that they can claim any integer amount 
between R and M {R and M are assumed to be positive integers 
with M>R). He also informs them that if they both ask for the 
same amount, they will be paid that amount, and if they ask for 
different amounts, each will be reimbursed at the lower value, but 
with a penalty R deducted from the higher claimant (who is 
assumed to have lied) and given to the lower claimant (as a reward 
for being honest). Thus, the TD game is a 2-person game with the 
discrete strategy set S = {R,R+\,R + 2, . . . ,M}, and payoff to an 
/-claimant against a y-claimant is defined by 
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i + R if/<7 
P(iJ) = { i if/ =7 (1) 
j-R ifi>j. 

In this context a Nash equilibrium is a pair of claims, such that, 
if each claim is known to the other traveler then neither has reason 
to revise their claim. For _R > 1 , there is an incentive for each 
traveler to undercut any common claim. Using backward 
induction, it is not hard to see that the travelers should each 
claim the amount R, i.e., (R,R) is a unique Nash equilibrium for 
the TD game. Thus, the unique Nash equilibrium of the TD game 
is the paradoxical outcome in which both travelers claim the 
lowest possible amount. We note, in particular, that the Nash 
equilibrium is independent of the reward/ punishment parameter 
R. However, as intuition would suggest, this is not how individuals 
actually play this game [17,19-22]. For instance, [22] found the 
following results when they played the game with 50 subjects (25 
pairs). The subjects could make claims between 180 and 300, in 
two treatments, one with i?=180 and another with R = 5. The 
results are shown in Figure 1(a). In the high-i^ treatment, close to 
80 percent of all the subjects chose the Nash equilibrium strategy, 
with an average claim of 20 1 . However, in the low-i^ treatment, 
roughly the same fraction chose the highest possible claim, with an 
average value of 280. Since the unique Nash equilibrium 
prediction is independent of the parameter R, classical game 
theory is unable to explain the most salient feature of these 
experimental results, namely, the effect of the reward/ punishment 
parameter R on average claim levels. 

The MEG game has a somewhat similar flavor to the TD game, 
in that the payoffs are again determined by the minimum of two 
actions [17,18,20]. In this game the players choose integer effort 
levels between 1 and M (where M is assumed to be an integer 
greater than 1), and a player's payoff is given by the minimum of 
the two effort levels minus the cost of the player's own effort. 
Therefore, the MEG game is a 2-person game with the discrete 
strategy set 5 = { 1,2, . . . ,M}, and payoff to an /-strategist against 
a j-strategist defined by 



P(iJ)= minOV)- 



(2) 



where c < 1 is a cost parameter. The MEG game suffers from the 
opposite problem to that of the TD game. Instead of exhibiting a 
single, deficient, Nash equilibrium, the MEG game exhibits 
multiple Nash equilibria; it is easy to see that any common effort 
level is a Nash equilibrium. Moreover, standard refinements of the 
Nash equilibrium concept do not select a subset of the equilibria. 
For instance, the Nash equilibria are strict, and thus trembling 
hand-perfect. Hence, classical game theory provides no obvious 
criterion to choose among them. 

As with the TD game, when the MEG game is actually played 
with human subjects the observed behavior is inconsistent with the 
results predicted by game theory [17,18]. For example, [17] found 
the following results in their experiment. The subjects could chose 
integer effort levels from 110 to 170, in one of two treatments, a 
low effort cost treatment of c = 0.1 and a high effort cost treatment 
of c = 0.9. The results are shown in Figure 1(b). In the low effort 
cost treatment the behavior is concentrated close to the highest 
effort level of 1 70, while in the high effort cost treatment the 
preponderance of the effort levels are at the lowest possible value. 
These results clearly indicate that the effort levels employed by 
subjects are inversely related to the effort costs, despite the fact that 
any common effort level is a Nash equilibrium. 

The paradoxical results obtained for the TD and MEG games 
using classical game theory are not resolved by instead using 
standard deterministic evolutionary game theory. Since the unique 
Nash equilibrium {R,R) in the TD game is strict it is a globally 
stable equilibrium point for the replicator dynamics [23]. Hence, 
the replicator dynamics of the TD game will always converge to 
the minimum claim level R. Similarly, in the MEG game every 
common effort level is a strict Nash equilibrium and hence a stable 
equilibrium point for the replicator equations. Thus, the behavior 
of the rephcator dynamics does not select any subset of the Nash 
equilibria. The paradoxical nature of both games is, therefore, 
equally evident when studied using either classical game theory or 
deterministic evolutionary game theory. 

It is noteworthy that the importance of the TD game and the 
MEG game are rather similar in nature. The TD game is 
theoretically significant because it exposes so clearly an apparently 
paradoxical aspect of game theory: namely the inability of tlie 
Nash equilibrium concept to predict the actual behavior of 
individuals interacting in this type of game. Moreover, it has been 
observed in [24] that the TD models competitive egg ejection in a 
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Figure 1. Results from playing the TD and MEC games with human subjects, adapted from [17]. (a) TD game: individuals make higher 
claims when the reward/punishment parameter R is low, and make lower claims when R is high, (b) MEC game: individuals expend more effort when 
the effort cost c is low, and less effort when c is high. 
doi:1 0.1 371/journal.pone.0093988.g001 
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species of communally nesting birds, the Greater Ani [25,26]. In 
this species, if two females share a nest then each female chooses a 
time to change from ejecting eggs from the nest to laying eggs. If 
both select an early time (which corresponds to a large claim), then 
both obtain a large payoff since they can both successfully lay 
many eggs. If, however, one chooses to wait, then she can eject the 
other birds already-laid eggs and obtain an even greater payoff, 
while at the same time inflicting a loss on the other bird [24]. 
Thus, this situation has the structure of the TD game. 

The MEC game also has both theoretical and practical 
importance. It is theoretically significant because it starkly 
illustrates the lack of prescriptive or predictive power inherent in 
the notion of Nash equilibrium. In addition, it has considerable 
practical import since many interesting and important real-world 
situations can be modeled by MEC games [27]. 

As an example of how MEC games naturally arise let us 
consider two companies, denoted by A and B, each of which 
manufacture a critical component of a jointly produced product. 
Let us suppose that company A makes widgets, while company B 
makes grommets. The final product, containing both a widget and 
a grommet, is sold jointly, with the revenues being equally split 
between the two companies. Each company can choose the 
amount of effort to expend in producing its component, with 
higher effort levels resulting in components of higher quality. We 
shall assume that the performance of the final product, and thus 
also the revenues obtained from sales of the product, is limited by 
whichever of the two components has the lower quality. 
Therefore, the profits obtained by a given company from sales 
of the product may be expressed as the minimum of the efforts 
expended by each company to produce widgets or grommets, 
respectively, minus the cost associated with the companies' own 
effort. Thus, such a situation can be modeied by a MEC game. 

The importance of the TD and MEC games, both theoreticsilly 
and in practice, is clear from the above comments. Therefore, 
obtaining a satisfactory understanding of the dynamics of these 
games is of considerable significance, and it is the purpose of this 
paper to contribute to such an understanding. 

A rnuTiber of different theoretical approaches have been 
investigated as possible explanations of the behavior found 
empirically in the TD and MEC games. In one approach, 
stochastic learning models [17,20,28] have been proposed to 
explain the anomalous behavior obserx^ed in the TD and MEC 
games. A quite different approach, using stochastic evolutionary 
dynamics in finite populations [24], has also been investigated as a 
means of resolving the paradoxical features of the TD game. 
Other theoretical approaches to explaining the behavior of the TD 
game have been studied in [29-32]. Other approaches which 
explore how c-rrors in a game may lead to deviations from Nash 
equilibrium play include [33-37]. 

Here we propose an alternative, considerably simpler, theoret- 
ical framework to explain the evolutionary dynamics of both the 
TD and MEC games, which accounts for the empirically observed 
behavior. Our method applies to a wide class of games that 
includes both the TD and MEC games. We first observe that while 
the TD and MEC games were originally formulated as discrete 
strategy games it is natural to consider variants of these games in 
which the strategies are continuously variable. In this paper we 
define these continuous-strategy variants of the TD and MEC 
games, which suffer from the same paradoxical behavior as the 
original discrete-strategy games, and it is these continuous-strategy 
games that form the starting point for our approach to 
understanding the evolutionary dynamics of the TD and MEC 
games. 



The continuous-strategy forms of the TD and MEC games are 
two examples of a large class of continuous-strategy games which 
have a discontinuous payoff function — other important examples 
of games from this class include the Bertrand Duopoly game [38] 
and the War of Attrition game [6]. For such games the role played 
by errors is potentially important. To be precise, let us consider the 
class of continuous-strategy games with payoff function given by 



Pix,y)=< 



(j>{x,y) 



-[(l>ix,y) + \l/(x,y)\ if x=y 



if x<y 
if x=y 
if x>y. 



(3) 



where (l>(x,y) and il/{x,y) are affme functions, and the strategies 
x,yeU.+ . This class of games includes the continuous-strategy 
variants of both the TD and MEC games. Errors in the 
observation of an opponents strategy or in the implementation 
of ones own strategy wiU result in the expected payoff to an x- 
strategist against a ^'-strategist in such a game being given by a 
function of the form 



Pix,y) = <t>(x,y)e(y -x) + il/ix,y)e(x-y). 



(4) 



where 6(y — x) defines the probability that in an interaction 
between an x-strategist and a j-strategist errors lead to the x- 
strategist receiving the payoff (j>(x,y), and 6{x—y) defines the 

probabilit)' that in such an interaction errors result in the x- 
strategist recei\'ing the payoff ij/(x,}'). We observe that the 
probabilit)' functions 6(x—y) and 6{y — x) are necessarily 
complementary in the sense that 0(x—y)-\-9(y — x)=l, from 

which it follows that 9{0)= -. The probability functions 6(x—y) 

and 9{y — x) are determined by the statistical distribution of errors 
in the game. Here, for simplicity, we shall assume that the function 
9 is smooth. Thus, the expected payoff (4) defines a smoothing of the 
original discontinuous payoff function (3). We shall often refer to 
the function 9 as the smoothing function. In the limit in which the 
smoothing function tends to the Heaviside step function the 
expected payoff (4) approaches the payoff (3). 

Since the purpose of this paper is to study the evolutionary 
behavior of the TD and MEC games it is necessary to define a 
suitable evolutionary dynamics for the class of games that we are 
considering. Since the effect of errors in the game is to give an 
expected payoff which is a smooth function, the simplest choice of 
dynamics is the standard deterministic adaptive dynamics [39-41] 
of the smoothed payoff function. In our approach the effects of 
errors in the game is encoded in the smoothing function 9. Here 
we study the evolutionary dynamics of the smoothed versions of 
the TD and MEC game, for an arbitrary smoothing function, and 
show that the results obtained are consistent with those found 
empirically. 

The approach to studying the evolutionary dynamics of the TD 
and MEC games that we follow in this paper has some advantages 
compared to stochastic learning models [17,20,28] or stochastic 
evolutionary dynamics [24]. In the case of stochastic learning 
models, the evolutionary dynamics is governed by the Fokker- 
Planck equation, which is a nonlinear partial differential equation 
that cannot be solved analytically [17, 20, 28]. The equilibrium 
solutions of the evolutionary dynamics are given by the solutions to 
a suitable differential equation. However, it is difficult to 
determine analytically whether or not the equilibrium solutions 
are stable. In fact, the equilibrium solutions to the stochastic 
learning models [17,20,28] for both the TD and MEC games have 
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not been shown to be stable, and thus, it is unclear whether or not 
they are attractors of the evolutionary dynamics. 

Stochastic evolutionary dynamics [24] represents an interesting 
alternative approach to understanding the dynamics of many 
evolutionary processes. In this case, the evolutionary dynamics is 
governed by a stochastic process, and the theory is mathematically 
well-developed. However, there are certain restrictions that apply 
to the theor)'. Perhaps the most important is that analytic results 
can only be obtained in the limit that the selection strength tends 
to zero. This limit corresponds to the assumption that the 
contribution to the total payoff that comes from the game 
interactions is very small. It is not clear that this assumption is 
realistic when it comes to understanding the behavior of the TD 
and MEC games. If the assumption of weak selection is not made 
then it is impossible to obtain analytic results, although numerical 
simulations can yield results for stronger selection strengths. It is 
also worth noting that while the stochastic process underlying 
stochastic evolutionary dynamics is mathematically well-devel- 
oped, it is rather subtle, and this can present a challenge when 
applying the method to new problems. In fact, stochastic 
evolutionar)' dynamics has not yet been applied to study the 
MEC game, although we conjecture that such an application wiU 
yield results consistent with the behavior observed in the game. 

A key advantage of the method that we propose here is that the 
evolutionary dynamics is much easier to study than for either 
stochastic learning models [17,20,28] or stochastic evolutionary 
dynamics [24]. In particular, it is straightforward to completely 
determine the evolutionary attractors for both the TD and MEC 
games. A second advantage of our method is that it applies direcdy 
to a wide variety of continuous-strategy games with discontinuous 
payoff functions, including the Bertrand Duopoly model [38] and 
the War of Attrition game [6]. In certain cases, which are 
considered at greater length in the Discussion, it can be shown 
using our methods that complex evolutionary dynamics, such as 
evolutionary branching, can occur in such games. 

The rest of the paper is organized as follows. In the Models 
section, we define continuous-strategy versions of the TD and 
MEC games, and also introduce the key notion of smoothed 
versions of these games. In the Analysis section, we analyze the 
evolution of strategies in the smoothed games in randomly- 
interacting populations using adaptive dynamics, and in addition 
formulate an agent-based model of the evolutionary dynamics of 
these games in populations with structures described by an 
arbitrary graph (i.e., network). In the Results section, we present 
the results of simulations using this agent-based model for the 
evolutionary dynamics of the smoothed TD and MEC games, 
both in well-mixed populations and in populations described by 
complex networks. Finally, in Discussion section, we provide a 
brief discussion of our work and draw some conclusions. 

Models 



<l>ix,y) 



if x<y 



- [(l>ix,y) + \l/(x,y)\ if x=y 



(5) 



ij/(x,y) 



if x>y. 



for affme functions (j>(x,y) and \l/(x,y), and strategies x,yeR+. We 
may write P(x,y) more succinctly with the aid of the Heaviside 
step function 0(m) 



©(«) = 



0 ifM<0 

1 ifu = 0 
2 

1 if M>0, 



(6) 



P{x,y) = <l>{x,y)@(y -x) + ij/{x,y)@{x-y). 



(7) 



Games of this form have discontinuous payoff functions. Such a 
discontinuous payoff function is only possible in an idealized world 
free from all errors. In reality, errors in the perception and 
implementation of actions in the game wiU have the effect of 
replacing the discontinuous payoff function with a smoothed 
approximation, representing the expected payoff. We now define 
such a variant of the game in which the discontinuity in the payoff 
function is removed by a smoothing procedure. To accomplish this 
we introduce a 1 -parameter family of smoothing functions 6i^(u). 
The functions Qk(u) are assumed to be smooth, non-decreasing 
functions of u, with lim„_, + oo 9ki>^)= 1, lini„^_oo df;(u) = 0, and 

(9/t(0)=-. Furthermore, we assume that 6k{u)-*&{u) as k—>-co. 

We will refer to the parameter k as the smoothing parameter. 

To obtain the smoothed version of the game defined by (7) we 
simply replace @(u) in the payoff function with its smooth 
approximation 9kiu). Thus, the payoff function of the smoothed 
game is given by 



Pkix,y) = (l>ix,y)9k(y -x) + \l/(x,y)9k{x - y). 



(8) 



We note that for suflicientiy large values of k the smoothed 
game approximates the original game arbitrarily well. 

A convenient 1 -parameter family of smoothing functions is 
given by 



0k(x) = 



1 



-kx' 



(9) 



and we shall use this family when explicit smoothing functions are 
required. 



The strategies in the TD and the MEC games are the claim 
levels and the effort levels, respectively. Typically these games are 
taken to have a discrete set of strategies. However, it is in many 
ways more natural to view the claim levels and effort levels in the 
two games as being continuously variable, and thus to consider 
variants of these games defined for continuous strategies. The 
continuous forms of the TD and MEC games are examples of a 
broad class of continuous-strategy 2-person games with payoff 
functions given by 



Traveler's Dilemma Game 

The claims made by individuals in the TD game represent their 
strategies. If x and y denote the strategies used by two individuals 
playing the continuous version of the game, and if R denotes the 
reward/punishment parameter (where x,y,ReU+), then the payoff 
to the x-strategist is given by 



P(x,y) = (x + R)®(y -x) + (y- R)@(x-y). 



(10) 



We now define a variant of the TD game defined by (10), in 
which the discontinuity in the payoff function is removed by the 
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smoothing procedure described above. To obtain the smoothed 
TD game we simply replace @{x) in the payoff function with its 
smooth approximation 6k{x). We therefore have that the payoff 
function for the smoothed TD game is given by 



Pk(x,y) = (x + R)ek(y-x) + (y-R)ek(x-y). 



(11) 



We shall assume, without loss of generality, that the strategy 
space in the smoothed TD game is the inter\'al [0,1], and also that 
the reward/punishment parameter _R6(0,I). With the payoff 
function defined by (11), the smoothed TD game represents a 
natural variant of the original TD game. 

Minimum Effort Coordination Game 

In the MEC game, the effort levels of the individuals represent 
their strategies. In the continuous version of the MEC game, if the 

strategies of two individuals playing the game are x and y (where 
x,yeU+) and c 6(0,1) is the effort cost, then the payoff P{x,y) to 
the x-strategist is given by 



P(x,y) = min {x,y) — c-x. 



(12) 



Using (12) allows us to write the payoff function for the MEC 
game as 



dynamics, if ^|^^^, <0. If this equality is reversed, x* is a 
repeller. 

Initially, the population will converge to a convergent stable 
singular point x*, but its subsequent evolutionary fate depends on 
whether x* is a maximum or minimum of the invasion fitness 

fxiy)- If X* is a maximum, i.e., if -^-j- Ij,^;^. <0, then x* is an 

evolutionarily stable strategy (ESS), representing an evolutionary 
end state in which all individuals adopt strategy x* . If, however, 

^— 5-L=j.» >0, then a population of x*-strategists can be invaded 

dy ' 

by mutant strategies on either side of x* . In this case the 
population undergoes evolutionary branching and splits into two 
distinct and diverging clusters of strategies. 

The adaptive dynamics of smoothed games with payoff fimction 
defined by (8) may be analyzed as follows. The invasion fitness is 
given by 



fx{y) = Pk(y,x) - Pkix,x) 

= <l>(y,x)6kix-y) + il/(y,x)ek(y - x) - + il/ix,x)] OkiO). 
Thus, the selection gradient D{x) is given by 



(15) 



P(x,y) = ix-cx)@(y-x) + (y-cx)&{x-y). (13) D(x)-- 



8fx(y) . 
dy ' 



Without loss of generaUty we can take the strategy space to be 
the unit interval (i.e. Xjj'Sp,!]). Every strategy pair (x,x) is a Nash 
equilibrium in this game. The social dilemma embodied in this 
continuous-strategv' game is clearly the same as for the original 
discrete MEC game: at any equilibrium both players obtain a 
payoff of (1 — c)x, thus all equilibria with the sole exception of the 
strategy pair (1,1) are Hicks inefficient. 

To obtain the smoothed MEC game we again replace 0(x) in 
the payoff function (13) with its smooth approximation 6kix). The 
payoff function of the smoothed MEC game is therefore given by 

Pk(x,y) = (x- cx)ek(y-x) + (y- cx)0k(x-y). (14) 

With the payoff function defined by (14), the smoothed MEC 
game represents a natural variant of the original MEC game. 

Analysis 

The dynamics of the smoothed TD and MEC games as 
formulated in the previous section can be analyzed in a well-mixed 
population using the deterministic framework of adaptive dynam- 
ics [11,39^1]. Consider a monomorphic population in which 
every individual adopts the same strategy, x. It follows from 
replicator dynamics that the growth rate of a rare mutant strategy, 
y, in the resident x population is fx(y) = Pk(y,x) — Pk(x,x), where 
Pkix,y) is the payoff to an x-strategist interacting with a y- 
strategist. The quantity fx(y) is referred to as the invasion fitness. 
The evolution of the strategy x is then governed by the selection 
8fx 

gradient -D(x) = ^— \y=x' adaptive dynamics of x is 

determined by the differential equation x = D{x). 

Equilibrium points of the adaptive dynamics are called singular 
strategies and are solutions of -D(x*) = 0. If no such solutions exist, 
then the strategy x monotonically increases or decreases under 
evolution, depending on the sign of D(x). If x* exists, it is 
convergent stable and, hence an attractor for the adaptive 



= ^ [<l>(y,x)9kix-y) + il/(y,x)ek(y - x) - (^(x,x)0a:(O) 

By 

-ik(x,x)ekm\y^x (16) 
= [Hx,x) - ^ix,x)]9',{0) + em [^(y,x) + ^(j,x)] |^^, 

= [iA(x,x)-0(x,x)]0;^(O)+ ^-^ [<^(y,x) + il,(y,x%^,. 

The adaptive dynamics of such a game is therefore determined 
by the differential equation 



x= [^(x,x)-<^(x,x)](?;(0)+ ~ [<l>(y,x) + ^{y,x%^,. (17) 

The existence of singular strategies x* in games of this form, 
and the particular characteristics of any such x*, depend on fxiy) 
and D(x), and thus ultimately on the specific functions 0(x,j') and 
i/'(x,7). We shall now apply these results to the TD and MEC 
games. 

Adaptive Dynamics of the Traveler's Dilemma Game 

Let us first analyze the TD game with the payoff function given 
by (1 1). Consider a monomorphic population of x strategists, i.e., a 
population in which every individual claims amount x. It follows 
from (2) and (11) that for the TD game (l>(x,y) = x + R and 
il/(x,y)=y — R. Thus, the selection gradient D{x) is given by 



Dix)=^--2Re'(0), 



(18) 



and the adaptive dynamics of x is consequently determined by 
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x=--2Re'(G). 



(19) 



Since D(x) does not depend on x, there are no singular 
strategies, and thus there is no possibility of exotic evolutionary 
outcomes, such as evolutionary branching. The evolutionary 
dynamics of an initial strategy x is determined by the sign of Z>(x). 

If D{x)<0 (i.e., it R> — ), then x will evolve to 0. If on the 
4^*4(0) 

other hand, D(x)>Q (i.e., — -, ) then x will evolve to 1. 

4l9/j(0) 

Thus, this adaptive dynamics analysis implies that the players of 
the smoothed TD game will evolve to make low claims if 

R> , , and, conversely, evolve to make high claims if 

R< , . We note that for the 1 -parameter family of smoothing 

functions defined by (9), this criterion takes the following form: 

claims wiU evolve to low levels if J?> ^, and evolve to high levels if 
1 

Adaptive Dynamics of the Minimum Effort Coordination 
Game 

We next turn to the MEC game where the payoff function is 
given by (14). Consider a monomorphic population of x strategists, 
i.e., a population in which every individual puts in x amount of 
effort. It follows from (2) and (13) that for the MEC game 
(l>{x,y) = x — cx and il/{x,y)=y — cx. Thus, the selection gradient 
D{x) is given by 



Dix)=^--c, 

and the adaptive dynamics of x is therefore determined by 



(20) 



studied both for random interactions between members of the 
population and for more complex interaction patterns in the 
population. The evolutionary dynamics of simple social dilemmas, 
such as the Prisoner's Dilemma and the Snowdrift game, have 
been well-studied for populations with a variety of complex 
interaction patterns [42-50]. 

Consider a population consisting of n individuals, labeled 
i=\, Since we wish to allow the possibility of complex 

population structures, we identify the population with the set of 
vertices in a graph F. The structure of F determines which 
individuals in the population can interact. Stricdy speaking, two 
graphs are required to specify the evolutionary dynamics: an 
interaction graph, F/, specifies that two individuals in the 
population can interact by playing the game only if they are 
adjacent in F/, and an updating graph, Fj/, specifies that an 
individual in the population can update its strategy by comparing 
its state to the states only of those individuals adjacent to it in F[/. 
Here, for simplicity, we shall assume that the interaction and 
updating graphs are the same i.e., F/ = Fj/ = F. Given an 
individual ieT, the set of neighbors of i (i.e., the set of individuals 
adjacent to / in F) wiU be denoted by N(i). 

The agent-based model is defined as a stochastic process on F. 
Let us fix either the TD or the MEC game as the game under 
consideration. We begin with a monomorphic population, i.e., 
each individual in the population starts out with the same initial 
strategy randomly picked from a uniform distribution. At each 
time step t, we carry out a round of asynchronous interactions 
followed by a round of asynchronous updates. Each of these 
rounds involves sampling the population with replacement. 

During an interaction step, we randomly pick an individual ieT 
and an individual jeN(i), and let the two individuals play the game 
against each other. If x and y denote the strategies of / and j, 
respectively, then the payoff P, received by the focal individual i is 
given by either equation (1 1) or (14), depending on the game under 
consideration. This procedure is repeated n times. 

During an update step, we randomly pick an individual feF and 
an individual jeN(i). If P, and Pj denote the payoffs of / and j, 
respectively, then with probability Pi^j given by 



x = - —c. 
2 



(21) 



1 



Again, since D(x) does not depend on x, there are no singular 
strategies. Also, rather remarkably, the adaptive dynamics of x is 
independent of the smoothing function B/i. The evolution of an 
initial strategy x is once again determined by the sign of D{x). If 

D(x)<0 (i.e., if c> ^) then x will evolve to 0, and ifD(x)>0 (i.e., 

if c < -) then x will evolve to 1 . Therefore, this adaptive dynamics 
analysis implies that the players' strategies in the smoothed MEC 

game will evolve to low efforts if the effort cost c is greater than -, 
1 

and to high efforts if c < -. Since the adaptive dynamics of the 

smoothed MEC game is independent of the smoothing function, 
these results hold for any smoothing of the game. 

We note that the behavior predicted by adaptive dynamics for 
the smoothed TD and MEC games is in accord with that observed 
for the TD and MEC games in experiments. 

Agent-Based Simulations 

In this section, we define a stochastic agent-based model which 
allows the evolutionary dynamics of the TD and MEC games to be 



l+e 



(22) 



the focal individual / will inherit fs. strategy. This update rule is 
often referred to as the Fermi rule. The parameter j6>0 is the 
"selection strength" of the update rule. The update procedure is 
repeated n times. 

Mutations are incorporated in the update procedure in the 
following way: when according to the update rule (22) i's strategy 
would be replaced by fi, then with probability /i, /'s strategy is 
instead replaced by a strategy picked randomly from a normal 
distribution with mean equal to y's strategy and standard deviation 
a. Carrying out n interaction steps followed by n update steps 
constitutes a single generation of the evolutionary dynamics. 

We note here that the results of our agent-based simulations 
(described in detail in the next section) are robust to variations in 
the update rule. For example, in addition to employing the Fermi 
update rule (22), we have also simulated the agent-based model 
using the replicator update rule, in which the probabihty that 
the focal individual feF inherits individual j's. strategy (with jeNQ)) 
is given by 
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0 \iPi>Pj 

m-Pi) 



M—m 



otherwise, 



(23) 



where M=maxPk, and w=minPfc. We find that the evolu- 

ter ter 

tionary dynamics of the smoothed TD and MEC games is the 
same irrespective of which of these update rules we employ. The 
results presented in the next section on the evolutionary dynamics 
of the TD and MEC games arise from simulations using the Fermi 

update rule (22). 

Results 

In this section we present the results of agent-based simulations 
for the TD and MEC games. For both the smoothed TD and 
MEC games the agent-based model described in the prc\-i()u,s 
section was simulated (using the Fermi update rule (22) and the 
smoothing function (9)) on the following graphs (see, for example, 
[51-54].): a complete graph (which models a randomly-interacting 
population); a random regular graph of degree 10; a scale-free 
graph with mean degree 10; and two-dimensional lattice graphs 
with 4 and 8 neighbors, respectively, and periodic boundary 
conditions. The games were simulated for 20000 generations. The 
parameter values used for the simulations were: population size, 
n = 1024 for the lattice graphs and n = 1000 for the other graphs; 
mutation rate, ^ = 0.05; standard deviation for mutations, 
(7 = 0.005; smoothing parameter, A: =1.5,2,3 for the TD game 
and A; = 0. 5,1,2 for the MEC game; and selection strength, fi = 5. 

Traveler's Dilemma Game 

Figure 2 shows the variation of the average claims x made by 
individuals over the last 20% of 20000 generations with the 

reward/punishment parameter R, for different values of the 
smoothing parameter k, on the following graphs: (a) a complete 
graph, (c) a random regular graph, (d) a scale-free graph, (e) a 2D 
lattice graph with 4 neighbors, and (1) a 2D lattice graph with 8 
neighbors. The R value was varied from 0 to 1 in steps of 0.05, 
and each data point was obtained from an average of 10 runs of 
the model. It is apparent from these results that, for each value of 

k, the claims are high when R<j and low when R> j, exactly as 

predicted by the adaptive dynamics analysis. This behavior is in 
good qualitative agreement with the results obtained for the TD 
game in experiments. Furthermore, these simulation results 
suggest that network structure has very little effect on the 
evolutionary dynamics of the game. 

Figure 2(b) shows the variation in the number of individuals 
with the claims x they make, when J? = 0.15 and R = 0.S5, on a 
complete graph, with k = 2. Individuals make higher claims when 
ii = 0.15, as indicated by the blue bars, and make lower claims 
when i^ = 0.85, as indicated by the green bars. This result is in 
good agreement with the empirical results shown in Figure 1(a). 

We have also simulated the game using the discontinuous form 
of the payoff function (10), and the results are shown in blue 
(labeled k=co) in panels (a), (c), (d), (e), and (f) of Figure 2. In this 
case, the claims are consistently low for all values of R. This result, 
which is consistent with the prediction of the adaptive dynamics 
analysis of the smoothed TD game in the limit fc-»oo, is in 
agreement with the prediction of classical game theory for the 
original TD game. Thus, for the TD game studied here, the 
smoothing of the payoff function is necessary to explain the 
empirically observed behavior. 



Minimum Effort Coordination Game 

Figure 3 shows th(^ \ ariati()n of the average effort levels x of the 
individuals over the last 20% of 20000 generations with the effort 
cost parameter c, for various values of the smoothing parameter k, 
on the following graphs: (a) a complete graph, (c) a random regular 
graph, (d) a scale-free graph, (e) a 2D lattice graph with 4 
neighbors, and (f) a 2D lattice graph with 8 neighbors. The c value 
was varied from 0 to 1 in steps of 0.05, and each data point was 
obtained from an average over 10 runs of the model. It is clear 
from these results that, for each value of c, the effort levels are high 

when c< - and low when c> -, exactly as suggested by the 

adaptive dynamics analysis. These results are in good quahtative 
agreement with the empirically observed behavior in experiments. 
The results of these agent-based simulations clearly show the 
independence of the evolutionary dynamics of the smoothed MEC 
game on the smoothing function (i.e., the independence on k), 
which is predicted by the adaptive dynamics analysis. These results 
also suggest that the effect of network structure on the dynamics of 
the smoothed MEC game is negligible. 

Figure 3(b) shows the variation in the number of individuals 
with their effort levels x, when c = 0.15 and c = 0.85, on a 
complete network, with k = 2. Individuals expend more effort 
when c = 0.15, as indicated by the blue bars, and expend lower 
effort when c = 0.85, as indicated by the green bars. This result is 
in good agreement with the empirical results shown in Figure 1(b). 

We also simulated the model using the discontinuous form of 
the payoff function (13), and the result is shown in blue {k= oo) in 
plots (a), (c), (d), (e), and (f) of Figure 3. In this case, there is stiU a 
transition from high to low effort levels as the effort cost increases, 
however, it is typically not as sharp as for the smoothed game (with 
any value of A:). An intriguing exception to this pattern is found in 
the case of two-dimensional lattice graphs, for which the 
evolutionary dynamics of the game with discontinuous payoff 
function is essentially identical to that of the smoothed game (for 
any value of the smoothing parameter). We tentatively conjecture 
that the feature of lattice graphs which is responsible for this effect 
is that they possess non-trivial clustering coefficients ([51-54]) — 
in contrast to the other graphs we have considered, which have 
zero clustering. The potential effect of the clustering coeflicient on 
the evolutionary dynamics of the MEC game defined on graphs 
appears to be an interesting topic for future research. 

Discussion 

In this work we have proposed simple and natural continuous- 
strategy versions of the classical discrete-strategy TD and MEC 
games. We have modeled these games as continuous games with 
smooth payoff functions, where the smoothing accounts for the 
effects of errors in the perception and/ or implementation of 
individuals actions. The smoothed TD and MEC games can be 
effectively analyzed using adaptive dynamics, which shows that the 
predicted evolutionary dynamics of these games is in accord with 
the behavior observed in empirical studies of the TD and MEC 
games. In addition, we have studied the evolutionary dynamics of 
the smoothed TD and MEC games using agent-based simulations. 
These simulations have been performed both for populations of 
randomly-interacting agents and for populations with more 
complex interaction patterns, represented by graphs of varying 
topologies. These simulation results are in agreement both with 
the analytical adaptive dynamics results, and also with the 
experimentally observed behavior. 

For the smoothed TD game, we find both from the adaptive 
dynamics analysis and from the agent-based simulations, that 
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Figure 2. Results from simulating the TD game. (a)(c)(d)(e)(f) Average claims x in the smoothed TD game over the last 20% of 20000 
generations versus the reward/punishment parameter R for different values of the smoothing parameter k, on a complete graph (a), a random 
regular graph with degree 10 (c), a scale-free graph with mean degree 10 (d), 2D lattice graph with 4 neighbors (e), and 2D lattice graph with 8 
neighbors (f). Parameter values: « = 1024 for lattice networks and «= 1000 for other networks, /i = 0.05, (t = 0.005, and /i = 5. (b) Number of individuals 
versus their claims a-, when i? = 0.15 and 7^ = 0.85, on a complete graph with parameter values: «=1000, ^( = 0.05, (t = 0.005, /c = 2, and 11 = 5. 
doi:1 0.1 371/journal.pone.0093988.g002 



claims vary with the reward/ punishment parameter R in a fashion 
that is in excellent agreement with the empirically observed 
behavior: low values of R result in high claims and high values of 
R result in low claims. We recover the classical game theory result 
that claims in the TD game with discontinuous payoff remain low 
for all values of R by considering the limit in which the smoothing 
parameter fc— >oo. Different interaction patterns among the 



individuals playing the smoothed TD game, as represented by 
studying the game on graphs of different topologies, appears to 
have little effect on the evolutionary dynamics of the game. 

We find similarly satisfactory results for the smoothed MEC 
game. The analysis, both analytical and through simulations, again 
yields results in good agreement with experiment: high effort levels 
are found for low effort cost c and low effort levels occur for high 
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Figure 3. Results from simulating the MEC game. (a)(c)(d)(e)(f) Average effort levels x in the smoothed IVIEC game over the last 20% of 20000 
generations versus the effort cost parameter c for different values of the smoothing parameter k, on a complete graph (a), a random regular graph 
with degree 10 (c), a scale-free graph with mean degree 10 (d), 2D lattice graph with 4 neighbors (e), and 2D lattice graph with 8 neighbors (f). 
Parameter values: n = 1024 for lattice graphs and « = 1000 for other graphs, /( = 0.05, (t = 0.005, and ji = 5. (b) Number of individuals versus their effort 
levels X, when c = 0.15 and c = 0.85, on a complete graph with parameter values: n= 1000, /i = 0.05, o- = 0.005, /c = 2, and fi = 5. 
doi:1 0.1 371 /journal.pone.0093988.g003 



effort cost. We again find that when the smoothed MEG game is 
formulated on graphs of differing topology the topological type has 
no significant effect on the evolutionary dynamics of the game. 

The methods introduced in this paper are quite general and can 
be applied to a wide variety of continuous-strategy games with 
discontinuous payoff functions. Important examples of other 
games that can be fruitfully studied using these methods include 



the Bertrand Duopoly model [38] and the War of Attrition game 
[6]. Here we will only briefly discuss the application of our 
methods to these two games — detailed accounts will be given 
elsewhere. 

In the classical Bertrand Duopoly (BD) model [38] one 
considers the interactions between two frrms that produce a 
homogeneous product. The strategy of each frrm is the unit price 
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that they set for their product. It is assumed that customers will 
buy a quantity d(p) (where d(p) is the demand function) from the 
firm with the lower price p, and will buy nothing from the firm 
with the higher price. If both firms set the same price then it is 
assumed that demand is split equally between them. It is also 
assumed that both firms have the same marginal cost c. Thus, the 
payoff obtained by one firm in its interaction with the other is a 
discontinuous function of the difference between the prices set by 
the two firms. This discontinuous payoff function can be smoothed 
in exactly the manner described in this paper to yield a smoothed 
game. In this case the smoothing function represents the 
probability that customers buy the product from the firm with 
the lower price as opposed to the firm with the higher price. In the 
Umit that this probability tends to one, the smoothed game 
approaches the classical game. 

The smoothed BD model can be analyzed using adaptive 
dynamics just as we have done here for the TD and MEC games. 
It may be shown for an arbitrary smoothing, that given a linear 
demand function d(p) there exists a unique singular strategy p* (in 
the domain of interest), that is stricfly greater than the marginal 
cost c. Furthermore, it may also be shown that the singular 
strategy p* is always both convergent stable and an ESS. Thus, the 
evolutionary dynamics of the smoothed BD game results in the 
prices set by both firms converging to the level p*, which is strictiy 
greater than the marginal cost. At the evolutionary equilibrium p* 
both firms obtain a positive payoff Thus, the introdut tion of the 
smoothed BD model effectively resolves the Bertrand Paradox that 
occurs in the classical model. Moreover, our method extends to 
the case of an arbitrary number of firms and sheds light on the 
results obtained in [55]. 
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The second example that will be mentioned here is the War of 
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